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of basic research on computer-based learning in the field. Beginning with 
research funded by the National Science Foundation and conducted at the 
University of Illinois and collaborating institutions, nearly 900 references 
on the PLATO [R] system are listed in the ERIC educational research database. 
This paper summarizes the research base of PLATO [R] Learning's instructional 
software, and describes the theoretical grounding of the curricula in the 
theory of reading, mathematics, and instructional design. It describes the 
PLATO [R] independent evaluation program and summarizes effectiveness studies 
performed as part of the ongoing program. PLATO [R] 's instructional design 
standards are described with nine tables, covering the areas of general 
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Abstract 

Professional educators demand that their methods and instmctional resources be 
soundly based on research and grounded in theory. This paper summarizes the 
research base of PLATO Learning’s instmctional software, and describes the 
theoretical grounding of the curricula in the theory of reading, mathematics, and 
instmctional design. 

The table below summarizes the research base described here. 



Professional 

Standard 


PLATO Learning’s Research 


Independent 

evaluations 


13 studies by independent evaluators are 
summarized here. Evaluators are university-and 
research center-based experts in evaluation and 
technology. 


Standard 

Evaluation 

Designs 


A variety of standard study types are included: 

• Experimental Design (1 study) 

• Comparison Group Design (5 studies) 

• Pre/Post-test Designs with gains reported 
(6 studies) 

• Case study with ending achievement 
reported (1 study) 


Standardized 

Tests 


Almost all studies use standardized and published 
tests for achievement. 


Gains sustained 
over time 


The studies describe program cycles of less than 
30 hours to two semesters. 

Program durations from 4 weeks to 4 years with 
sustained gains across multiple program cycles. 



Professional 


PLATO Learning’s Research 


Standard 




FuU 

Implementations 


Each study represents a full implementation of the 
PLATO courseware, using one of three flexible 
implementation models which have been defined 
for PLATO: supplementary, complementary, or 
primary. 


Implementations 
well described 


Implementations are fully described in the 
complete evaluation reports. They are briefly 
summarized here. 


Costs well 
documented 


Cost data, including hardware, software, support, 
and professional development, are provided by 
PLATO Learning persoimel. 


Similar service 
populations 


The studies include urban, suburban, and rural 
contexts, many with underachieving, low-income 
and diverse populations. 



Professional 

Standard 


PLATO Learning’s Research 


Wide replication 


The studies describe replication of the PLATO 
system across a range of settings: 

• elementary, secondary, and post- 
secondary. 

• A variety of program types, including jh// 
implementations in primary uses of 
PLATO (mostly self-paced, mastery 
model, individualized with instructor 
tutoring and coaching), and partial 
implementations in supplementary uses 
(for review and reinforcement of 
classroom instruction), often with a goal of 
standardized test remediation or 
preparation. 

• Program sizes range from 25-1 ,000 
students 


Replications 

evaluated 


Each of the studies summarized here is supported 
by a full evaluation report. Many additional case 
studies (with abbreviated reports of data) are 
available on request. 



We can make these general observations from the studies: 



Improved 
performance on 
standardized 
tests 



The greatest gains are in programs using the primary 
instructional model for a semester or more (at least 30 
hours of use). Effect sizes, where reported, were up to 
1.5, or 2 standard deviations. This represents 
improvements of up to 60% on achievement of standards. 



Supplemental programs generally produce smaller gains, 
but one study reported gains of over 3 standard 
deviations. Effect sizes were most commonly under .5, 
representing improvements of up to 15% on achievement 
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of standards. 



Improved credit 
recovery and 
dropout 
prevention 


Improved graduation rates result in credit recovery and 
dropout prevention programs with up to 100% success 
rate. 


Improved time 
on task leads to 
achievement 
gains 


In the primary instructional model longer programs with 
more study of PLATO produce greater effects with 
significant correlations of up to +.83. While the 
relationships between time on task with PLATO and 
achievement are complex and isolation of the effects of 
PLATO is never a goal of these evaluations the 
relationship does suggest positive effects of PLATO. 



Full evaluation studies, PLATO Technical Papers, and additional implementation 
reports are available from the PLATO web site, www.plato.com . 



PLATO curricula are grounded in the theory of each major content area and the 
field of Instructional Design. 



In reading, the principal influence on PLATO’s curricula has been the research 
summarized recently by the National Reading Panel (NRP). The table below 
shows the relationship of the NRP’s conclusions and the PLATO elementary and 
secondary reading curricula. A more detailed discussion is in Part 3 of this paper, 
and in PLATO technical papers on reading. 



Key NRP 
Conclusion 


PLATO’S Curriculum Design 


7 cognitive strategies 
have been validated 
for teaching reading 
comprehension. They 
should be applied in 
combination to the 
reading task. 


These 7 cognitive strategies are incorporated in the 
PLATO secondary reading curriculum components 
which teach reading comprehension extensively. The 
strategies are taught as heuristics to be combined and 
applied to the reading task at hand. 

The same 7 strategies form the basis of the elementary 4- 
6 reading curriculum, now in development. 

In addition, reading is an essential skill for the 
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Key NRP 
Conclusion 


PLATO’S Curriculum Design 

interdisciplinary real-world problem solving activities 
which are part of the PLATO curricula. These activities 
provide real-world, motivating context and establish the 
need and occasion for high-level comprehension in a 
collaborative learning environment. 


Vocabulary 
instruction leads to 
gains, and should be 
taught both directly 
and indirectly. Use of 
computers was found 
to be more effective. 


PLATO’S curriculum includes both direct and indirect 
vocabulary instmction. 

Direct instruction is accomplished with the Vocabulary 
Builder system, which teaches pre-reading vocabulary 
and SAT vocabulary, and provides a convenient tool for 
teachers to build their own word lists. 

Indirect instruction is accomplished through 
incorporation of a level-appropriate fiill online dictionary 
which can provide definitions for any on-screen word or 
any word typed in. 


Fluency is a critical 
skill, and can be 
taught through a 
combination of 
guided repeated oral 
reading and silent 
reading 


Guided repeated oral reading is supported in PLATO 
curricula at the elementary and lower level secondary 
courses through read-the-screen audio. This feature can 
be turned on or off by the instructor. 

Silent reading is supported at all levels through short, 
medium and long passages of a variety of text types, all 
carefiilly leveled using standard reading formulas. 


Phonics instruction 
should include 
explicit, systematic 
phonics instruction 
and a focus on putting 
the letter-sound 
relations to use in 
reading whole words 
and passages. 


A fiill technology-based phonics solution is beyond the 
state of the art, and must be implemented by instructors. 
PLATO elementary curricula are designed to supplement 
a systematic phonics program, by providing additional 
practice on phonics as part of the task of reading whole 
words, sentences and passages. 


Phonemic Awareness 
instruction using 
explicit methods is the 
foundation of 
successful reading. 


A fiill technology-based phonemic awareness solution is 
beyond the state of the art, and must be implemented by 
instmctors. PLATO elementary curricula are designed to 
supplement a phonemic awareness program, by providing 
practice and reinforcement if PA at the grades 1-2 levels. 



In mathematics, the principal influence on PLATO’S curricula has been the 
research applied to develop the curriculum standards of the National Coucil of 
Teachers of Mathematics (NCTM) The table below shows the relationship of key 
trends found in the NCTM standards and the PLATO elementary and secondary 
mathematics curricula. A more detailed discussion is in Part 4 of this paper, and in 
PLATO technical papers on mathematics. 



NCTM Approach 


PLATO Mathematics Curricula 


Learner-based: 

The learner discovers and 
constructs meaning. The learner 
encounters the core concepts and 
principles through investigation. 
(The teacher provides opportunities 
for investigation and facilitates.) 


PLATO curricula support an investigation 
and problem-centered approach. If the 
instructor wishes, the modular curricula can 
be structured around a core of problem 
solving activities (both on- and offline) and 
math investigations. Learners can then 
work through the strong direct instructional 
components of the curriculum to master the 
declarative knowledge and well-structured 
procedures needed to construct meaning. 


Integration of math strands: 

algebra, geometry, data analysis, 
etc., taught each year. Coimections 
among math strands are explored. 


PLATO curricula are highly modular, and 
can be sequenced as the instructor desires. 
In addition, certain key concepts and skills, 
such as functions, are addressed in multiple 
levels to support a spiral structure. 


Problem/situation-based 

approach: 

Students learn concepts and 
principles as they explore a real- 
world problem or situation. 
Students use a wide range of what 
they know to solve rich problems. 


PLATO problem solving activities involve 
real-world scenarios of compelling interest 
to elementary, secondary and adult learners. 
The PSA’s require integration of many 
math strands, as well as integration of math 
with other curriculum knowledge. Work 
can be done in collaborative or individual 
mode. 


Function based approach: 

This goes hand-in-hand with the 
problem-based approach. Students 
observe real world functions early 
in the curricula and know the 
concept of function prior to 
learning formal notation and 
advanced concepts. 


Functions concepts are introduced early in 
the Algebra I curriculum, and treated at 
multiple levels on into Algebra n. 
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NCTM Approach 


PLATO Mathematics Curricula 


Emphasis on Representation - 
math as language: 

Emphasis on many ways to 
represent problems and many ways 
to solve them using various 
representations (which leads to 
multiple solution paths and 
sometimes more than one 
acceptable answer). 


Multiple representations are at the core of 
the PSA’s, which typically support 
representations in graphical, equation, and 
matrix form. The elementary curriculum a 
also adds manipulables as a form of 
representation. 


Emphasis on higher order 
thinking/process: 

Learners analyze, interpret, explain 
their reasoning. Learners generate 
algorithms. 


Analysis and explanation of reasoning 
processes are at the core of the PSA’s 
throughout the elementary and secondary 
math curricula, and in the Investigations 
activities in Algebra I and n. 


New topics (and at lower levels): 
Data collection and analysis, 
statistics, probability, and discrete 
math topics are taught 6-8 and 9-12. 


These topics are incorporated in the PSA’s 
as well as selected tutorial activities 
throughout the elementary and secondary 
curricula. 


Integration of information about 
the history of mathematics and 
its contributions. 


In the new secondary mathematics 
architecture, found in Algebra I and n, the 
history of mathematics is a key element of 
the Investigations activities. 


Technology integrated as tool to 
allow exploration of 
concepts/principles. 


In the elementary curricula, the Toolbox is 
available for free exploration of concepts 
and principles. 

In the secondary PSA’s, a Tool Bar 
provides similar access to appropriate free- 
play tools. 



Instructional Design is the theory base which applies to all PLATO curricula. The 
instructional design standards of PLATO are based on a current cognitive learning 
theory (Anderson’s ACT’" model), and apply current best practice instructional 
strategy recommendations for teaching of each type of declarative and procedural 
knowledge. Further details are in Part 5 of this paper. 
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Part 



Introduction 



From its origins nearly 40 years ago, the PLATO system has built the largest base 
of basic research on computer-based learning in the field. Beginning with 
research funded by the National Science Foundation and conducted at the 
University of Illinois and collaborating institutions, nearly 900 references on the 
PLATO system are listed in the ERIC educational research data base. PLATO 
Technical Paper #7(Foshay 1994) summarized meta-analyses of the effectiveness 
of computer-based instruction, and also summarized selected reports on PLATO 
courseware through 1993. 

This paper reviews three bodies of research relevant to the current PLATO 
Learning system. First, we will describe the PLATO independent evaluation 
program and summarize effectiveness studies performed as part of the ongoing 
program. Full-length evaluation reports are available from the PLATO Learning 
web site, www.Dlato.com . Next, we will provide an overview of the theory base 
in instructional design of PLATO courseware. Finally, we will summarize the 
theory base of the PLATO curricula in reading and mathematics. For greater 
detail on each of these curricula and their theory base, refer to the PLATO 
Technical Papers on reading and mathematics ((Foshay, McEvoy et al. 2000; 
Quinn, Foshay et al. 2000; Quinn, Foshay et al. 2000) also available from the 
PLATO web site. 

The PLATO system is unique in the industry for its grounding in theory, 
comprehensiveness, innovation and quality. All of the core curricular products in 
the PLATO system have been built internally by a curriculum development staff 
which has been in continuous operation since 1976, and has evolved its 
instructional design standards and methodologies to reflect the state of the art in 
theories of learning and instruction and the major curriculum fields. By the end of 
this year, over 70% of the courseware in the PLATO portfolio will have been built 
from scratch in the past five years, or will have received a major instmctional 
upgrade and expansion. PLATO products regularly receive awards for quality 
from universities, education magazines, and trade associations. PLATO 
courseware has been cited in recent textbooks on eLearning and instructional 
design, and has been the subject of many papers and presentations at academic 
research conferences in the past five years. As a result, PLATO has developed an 
enviable reputation as an exemplar of the state of the art in instructional design, 
curriculum theory, and learning and instruction, with impressive evidence of 
effectiveness with elementary, secondary, post-secondary and adult learners. 




The Research Base ofPLA TO 
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Part 









Research on Effectiveness of PLATO Curricula, 1994-2001 



The current federal education legislation (the No Child Left Behind Act of 2002) 
has spurred renewed interest in research-based methods in education. The issue of 
defining just what constitutes adequate evidence of “research based” practice is 
problematical. Foshay and Quinn (Foshay and Quinn in press) argue that the issue 
has numerous complexities: 

A common perception is that experimental designs are “more rigorous” than 
quasi-experimental designs. As with all aspects of evaluation design, 
however, there are tradeoffs to be considered when choosing between 
experimental and quasi-experimental designs. Generally speaking, 
experimental designs yield findings that are easier to interpret and more 
credible for making causal conclusions. However, there are a number of 
limitations to experimental designs. They are more expensive to implement, 
since special arrangements are often required. They are often more intmsive 
requiring the participant, teachers, and others involved with the training 
activity to adjust their schedules to meet the evaluator’s requirements. And 
there may be legal, policy, or natural constraints on the program being 
evaluated that make random assignment or other experimental conditions not 
possible. 

In addition to not being able to implement an experimental design, there are 
other times when quasi-experimental designs are more appropriate. The very 
fact that experiments are so intmsive may in and of itself change the situation 
so much that any results fi’om an experimental study are not valid for 
describing what is likely to occur in the natural environment for the program 
being evaluated. As participants become aware that they are being studied they 
act differently, sometimes quite differently, fi-om what might be observed in a 
more normal setting. In these cases it may be possible to identify existing 
measures that can be used, or even analyzed after the fact, using quasi- 
experimental designs. 

Accordingly, PLATO Learning has adopted an ambitious and comprehensive 
program of evaluation research on effectiveness of its products; what we believe to 
be the largest such ongoing program in the industry. Its primary goal is 
documentation and dissemination of “best practices” in PLATO usage. This 
knowledge is an important benefit for our clients, because it gives them concrete 
models to adapt and use in their settings. 




The Research Base ofPLA TO 
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We are systematically pursuing a plan to gather data from a wide range of sites 
and applications of PLATO, in order to be able to say that: 

• when PLATO is used in a given way 

• with a given target population 

• these are the results which have occurred, and 

• these are the external test results which have been measured 
This evaluation program applies these principles: 

• Different ways of using PLATO will lead to different results. Therefore, it is 
important to study a variety of ways of using PLATO, so that we will be able 
to make recommendations for expected gains under different usage scenarios. 

■ These studies thus examine all of the effects of the program, and make no 

attempt to experimentally or statistically isolate the effects of PLATO alone. 
Instead, the studies emphasize a thorough description of how PLATO is used 
and the context of its use, as well as reporting achievement data. PLATO 
Technical Paper #6, Instructional Models: Four Ways to Integrate PLATO 
Into the Curriculum 2000) distinguishes between three classes of use: 

supplementary, complementary, and primary. Most PLATO evaluations 
characterize the program studied according to these terms: 

Supplementary strategies use PLATO for review and reinforcement of what 
has already been taught by other means. 

Complementary strategies use PLATO to add new content to the curriculum, 
such as problem-based activities, enrichment, or remediation. 

Primary strategies use PLATO for initial teaching of parts of the curriculum. 

• As discussed above, there are substantial limitations on the usefulness of 
experimental and even control group studies. Therefore, we use a mix of 
quasi-experimental (comparison group) and descriptive (case study) 
evaluation designs, and follow standard methodological recommendations 
appropriate to the design. 

• All evaluations are performed by an independent evaluator. PLATO 
evaluations have been done by faculty members in instmctional technology at 
major universities, by independent evaluation consultants with backgrounds in 
university and educational research laboratory settings, and by the clients’ own 
evaluators. 

• All evaluations include at least a post-test of achievement using a recognized 
non-PLATO test (most often a state competency test or other standardized 

The Research Base ofPLA TO 4 
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test). Where possible, pre-test data also are obtained, and gain scores are 
reported along with their statistical significance. However, many educational 
programs do not have pretest data which precedes commencement of PLATO 
use closely enough to be of use in an evaluation. 

• One question that often arises when discussing mean differences between two 
groups is the size of that difference. A common measure for the difference 
between two group means is called effect size. Effect size is calculated by 
subtracting the smaller mean fi'om the larger mean and then dividing the result 
by the average of the two standard deviations for the means. In educational 
research an effect size of .75 or greater is usually considered large, fi'om .50 to 
.75 is moderately large, and fiom .25 to .50 is usually considered moderate to 
small. An effect size that is less than .25 standard deviations is usually 
considered too small to be educationally interesting. Where possible, PLATO 
evaluations report effect size. 

• There is no straightforward relationship between patterns of PLATO use and 
achievement, because of complexities of placement, progress, and various 
learner variables. As a result, we report utilization patterns and correlations of 
module mastery and time on task with achievement as a means of describing 
the program, but do not attribute causal significance to them. 

• There is no intent to “prove” that PLATO is (or is not) effective vs. classroom 
teaching or other media. Media comparison studies have been widely 
criticized in the professional literature as uninterpretable, and the same would 
be tme with any such study done with PLATO. PLATO is best conceived of 
as a tool which can enhance learning environments, not replace them. 

The studies summarized here have been completed over seven years in academic 
environments'. The full studies are available at www.plato.com. or on request. 
For each study, the summary reports the name of the site (if permission was 
obtained to use it), a description of the context, the type of instmctional model, 
achievement data and other quantitative results, and effect size (if it was reported). 
This summary is followed by a brief description of the program, and of other 
effects observed. A summary of additional reports completed fiom 1980-1993 is 
available in PLATO Technical Paper #1, Effectiveness of Computer-Based 
Training: An Annotated Bibliography of Reviews, 1980-1993 (Foshay 1994). 



^ Additional studies on JTPA and wor1<place sites, and a variety of program reports with qualitative data, are also 
available upon request. 



The Research Base ofPLA TO 
Copyright ©2002 by PLA TO Learning, Inc. 



5 






Summaiy of Independent Third-Party PLATO Evaluation Research Studies in 

Academic Contexts, 1993-2001 

13 studies by independent evaluators are summarized here, representing wide 
replication of the PLATO system across a range of implementations, each with an 
independent evaluation. Most use standardized and published tests for 
achievement. They include: 

• A variety of study types (1 experiment, 2 comparison group, 1 pre/post-test 
with gains reported, 1 case study with ending achievement reported) 

• A range of levels, including elementary, secondary, and post-secondary. 

• Service populations include urban, suburban, and rural, many with 
underachieving, low-income and diverse populations. 

• A variety of program types, including implementations in primary uses of 
PLATO (mostly self-paced, mastery model, individualized with instmctor 
tutoring and coaching), and partial implementations in supplementary uses 
(for review and reinforcement of classroom instmction), often with a goal of 
standardized test remediation or preparation. 

• Implementations are fully described in the complete evaluation reports. They 
are briefly described here. 

• Program sizes ranging fi-om 25-1 ,000 students 

• Program durations fi'om 4 weeks to 4 years with sustained gains across 
multiple program cycles. 

• Program cycles of less than 30 hours to two semesters 

Note that since these studies were completed, the PLATO secondary reading 
curriculum has been completely replaced, the mathematics curriculum has been 
expanded and upgraded, and all other major curricula are being upgraded, 
expanded or replaced in the near term. Future evaluations will include these new 
curricula. 

We can make these general observations fi"om the studies: 

• The greatest gains are in programs using the primary instmctional model for a 
semester or more (at least 30 hours of use). Effect sizes, where reported, were 
up to 1.5, or 2 standard deviations. This represents improvements of up to 
60% on achievement of standards. 

• Supplemental programs generally produce smaller gains, but one study 
reported gains of over 3 standard deviations. Effect sizes were most 
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commonly under .5, representing improvements of up to 30% on achievement 
of standards. 

• Improved graduation rates result in credit recovery and dropout prevention 
programs. The most effective programs reported credit recovery success for 
every participant. Pass rates on state exit exams ranged up to 85% in English, 
and 100% in Math. 

• In the primary instructional model, longer programs with more study of 
PLATO produce greater effects, with significant correlations of up to +.83. 
While the relationships between time on task with PLATO and achievement 
are complex, and isolation of the effects of PLATO is never a goal of these 
evaluations, the relationship does suggest positive effects of PLATO. 
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Houston Community College Mathematics 
Department 


Study Type: Experimental Study 


Developmental Studies Mathematics 


Context Type: Developmental studies math 
program, urban 2-year community college 


1,000 students in Fundamentals of 
Mathematics I 

• 46 students in the control group 


Program Duration: 

Experimental group (PLATO) averaged 27 
hours 


• 35 students in the experimental group 

Random assignment to the groups. 


Control group (classroom) had 48 hours in 
class with required homework 


Key Quantitative Results: 


Significance: /?=0.015 


• Mean score improvement for the 
experimental group was 7.8 vs. 5.2 


Effect Size: +.61 


for the control group 
• On pretest, experimental group scored 
a mean of 15.49 (s.d.=4.99); control 
group scored a mean of 14.91 
(s.d.=4.12), so the groups were 
equivalent. 


Measure Used: 

Pre/post test: Arithmetic and Basic Skills test 
of the Committee on Placement 
Examinations, The Placement Test Program 
of the Mathematical Association of America 


• On post-test, the experimental group 
showed a 24.4% gain, and the control 
group showed a 14.4% gain. 




• In a regression test, a beta of 0. 1 0 was 
attributed to the experimental group 

(p<.001) 




• Correlation of usage to gain was .27 
(p=.015) 





Instructional Model: 



Experimental group self-paced, mastery model, with instmctor tutoring. Control group used 
the same instmctors in class, with comparable curriculum. Curriculum ahgned to the TASP. 
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Fair Park HS, Shreveport, LA 

Secondary Math and English 

138 Students preparing for graduation 


Study Type: 
Comparison study 


Context Type: 

High school test preparation program 


Program Duration: 

1 year 


Key Quantitative Results: 

• In Mathematics, 79% of PLATO 
students passed, vs. 51% non- 
PLATO students 

• In English/Language Arts, 79% of 
PLATO students passed, vs. 67% 
non-PLATO students 


Significance: N/A 


Effect Size: N/A 


Measure Used: 

Louisiana Education Assessment Program 
(LEAP) 


Key Qualitative Results: 

• Teachers rated PLATO very highly and recommended expansion of the program. 

• Teachers reported students “loved it” and commented on self-paced review and 
advancement. 
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Lakeland HS, FL 

Secondary Math and Reading 
3 1 students (Math) 

29 students (Communication) 


Study Type: Case Study with pre/post test 
gains 

Context Type: 

Remedial lab for students who failed the 
FHSCT. 

Suburban HS uses PLATO labs for skill 
remediation, SAT preparation, and specific 
skill development, at all levels. 

46% free/reduced lunch, 24% African 
American, 6% Hispanic, 2.7% dropout rate. 




Program Duration :4 years (data for 2 years) 


Key Quantitative Results: 


Significance: 

/7<.001 (math) 


Year 2 results: 


/7<.001 (communication) 


• Fall-to-Spring gains in math averaged 


Effect Size: 


40 points, for an exit mean of 717 


1.40 (Math) 


points 

• Fall-to-Spring gains in English 


1.58 (English) 


averaged 18 points, for an exit mean of 
704 points. 


Measure Used: 


At the second retest. 


Florida High School Comptetency Test 


• In math, 100% of students passed. 


(FHSCT) 


• In English, 85% of students passed. 


pre-/post- test 


Instructional Model: 




Primary mastery model instmction with active teacher in “guide on the side”role 

Instructor Ratings: 

29/34 questions rated 4 or 5 out of 5 

Learner Ratings: 

15/20 questions had means above 4/5. 

5 had means above 3/5. 
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Apache Junction, AZ 

Elementary Reading, Math, Language Arts 
100 elementary students 
75 middle school students 


Study Type: Case study with pre/post gains 


Context Type: Grades K-8 (data fi'om grades 
1-4) 

Remedial reading, language arts & math 
Urban fiinge/rural, with white, Hispanic, 
American Indian, Asian, Black students 


Program Duration: 

4 week, 64 hour summer program 


Key Quantitative Results: 

• Grade 3 reading: average gain of 27% 

• Grade 3 math: average gain of 25% 

• Grades 2 & 3 final scores in reading & 
math ranged from 80% - 90% 

• Grades 1 & 4 final scores in reading & 
math ranged from 63% - 90% 


Significance: 

Correlation between PLATO use and 
achievement significant at /?<.001 
Gains significant at /K.OOl 


Effect Size: 
Reading: +2.16 s.d^ 
Math: +3.13 s.d. 


Measure Used: 

Locally developed reading, language arts & 
math tests; pre- and post test data available 
for 3rd grade. 


Instructional Model: 

Supplemental model; 25 minutes/day average use, average of 10-35 activities mastered 

Qualitative Results: 

Instructor Mean Ratings (5 point scale): 

Content 3.4 -3.9 
Instmctional Design 2.8-4.0 
Teacher experience 3. 0-3. 8 
Student experience 2.6 - 3.3 
PLATO Activities: 3.5 - 4.1 

Learner Ratings: N/A 



^ S.d = standard deviation 
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Jobs for Youth - Boston 

Madison Park Technical-Vocational HS, 

Boston, MA 

Secondary reading and mathematics 
185 students, 9* grade 

(Complete data for 77 in math, 47 in reading) 



Key Quantitative Results: 


Significance: N/A 


• Positive correlation between # 


Effect Size: 


Modules mastered and math post- 


Math: +1 s.d. 


test (r=.37,p<.05) 


Reading: +2 s.d. 


• Math: average gain of 19 points on 




O O X 

BPS math 


Measure Used: 


• Positive correlation between hours 




of PLATO use and reading gain 


Pre/post-tests : Boston Public Schools Math 


(r=.38,p<.05) 


(BPS) and Scholastic Reading Inventory 


• Reading: average gain of 266 Lexiles 


(SRI) 



Instructional Model: 

Supplementary use, 40-50 min of PLATO, 4 days/week, in 4 hour block of math and/or 
reading instmction. Average total of 9-12 hours on PLATO. 

Key Qualitative Results: 

• PLATO was well liked by teachers & students. 

• Learning curve and hardware issues 

• Assessments long 

Similarity of items in practice & tests (an advantage) may discourage use by teachers 
concerned with boredom 

Instructor Ratings: 

Mean ratings on all items 4 - 4.75/5, except training, 2.75. 

Learner Ratings: 

Mean ratings on all items above 3.5, except “I feel I’m studying what I need to” 2.4 



Study Type: Case study with pre/post gains 



Context Type: 

Urban underachieving students, many 
bilingual or non-English speakers (Spanish & 
Creole). Some learning disabilities, attention 
problems, problem homes 

Program Duration: 

5 wk summer remedial program for the 
lowest scoring students in the lowest-scoring 
school in the state. 
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Western Harnett HS, Lillington, NC 

Secondary Math & Reading 

25 students who failed the state competency 
test in November 


Study Type: Case study with pre/post gains 


Context Type: 

Rural HS with high military/transient 
population. 

Remedial program for students who failed the 
state competency test. 


Program Duration: 

1 semester 


Key Quantitative Results: 

• 60% of math students passed 

• 43% of reading students passed 

• Mean grade level gain of 1 .68 in 
math 

• Mean grade level gain of 2.87 in 
reading 


Significance: N/A 


Effect Size: N/A 


Measure Used: 

North Carolina Competency Test pre/post test 


Instructional Model: 

Primary instruction, individualized placement & self paced, with active instructor tutoring and 
coaching, and peer tutoring. In a 4 course/4 block per day schedule, learners used PLATO 2-3 
blocks per week, from February through the end of the school year. 

Key Qualitative Results: 

• Instructor were positively impressed with PLATO and expanded its use to other 
applications. 

• Peer tutors positive about PLATO. 

Learner Ratings: 

Peer tutors rated all questions with a mean of 4 or above. 
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RL. Turner HS, Carrollton, TX 

Secondary Math 

120 at-risk 10* grade students 

144 1 1* & 12* grade students who failed the 
TAAS 


Study Type: Comparison Study with 
pre/post gains 


Context Type: 

Diverse HS with 56% minorities, 38% 
Hispanic, 8% African American, 40% 
free/reduced lunch. 

Program targeted at risk 9* graders, and 10* 
- 1 1* graders who failed the TAAS. 


Program Duration: 2 years (math) 


Key Quantitative Results: 

• Pass rate in math improved from 
69% before the program to 83% 

• 74% of high-risk 10* graders 
passed the math TAAS. 

• 87.5% of at-risk 11* & 12* graders 
(who previously failed the TAAS) 
passed a re-attempt of the TAAS. 

• Overall school average pass rate for 
TAAS was 83% vs. 86% statewide. 


Significance: N/A 


Effect Size: N/A 


Measure Used: 

Texas Assessment of Academic Skills 
(TAAS) (pre- & post-test for 1 1* & 12* 
grade, post-test for 10* grade) 

TAAS practice test pre-test for 10* grade. 


Instructional Model: 

Primary instmction with active tutoring and counseling. Individualized prescription, self 
paced, mastery model. Pullout program, at least 1 hr/week PLATO use in 30 min. blocks. 

Key Qualitative Results: 

Faculty and principal were positive about PLATO and felt it contributed to program success. 
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Lawrence Central HS, Indianapolis, IN 

Secondary Math and English (reading & writing) 

406 students, 11* grade (complete data for 136 in Math, 
97 in English) 


Study Type: Case study with 
pre/post gains 


Context Type: 

Remedial program for students 
who failed the ISTEP. 

Suburban HS with mix of 
affluent, upper middle-class, blue 
collar, military, and poor learners. 
25% minority, 30% below grade 
level. 


Program Duration: Two years 


Key Quantitative Results: 

After 1 semester of study (year 1): 

• In math, mean score increased by 26 points 

• In English, mean scores increased by 25 points 
After 2 semesters of study (year 2): 

• In math, mean score increased by 36 points 

• In English, mean scores increased by 28 points 

• Of 406 students in the program, all but 74 
passed by the end of year 2. 

Success in the course (based on PLATO module mastery) 
was positively correlated with ISTEP score {r=.AA, 
jp<.001 for Fall, r=332,p=.02S for Spring). 


Significance: p.<.001 


Effect Size: N/A 


Measure Used: 

Indiana State Testing for 
Educational Progress (ISTEP) 


Instructional Model: 



Primary instruction with teacher actively coaching and tutoring. Individualized placement 
testing, prescription, self-paced, mastery model. Alternate 90 minute blocks in PLATO lab 
and in classroom. 

Key Qualitative Results: 

Math and English teachers very happy with PLATO 
Both attributed student success to PLATO 
Teachers commented on fewer discipline problems with PLATO 
Teachers noted need for a nurturing, involved lab manager 

Instructor Ratings: 

All questions but 5 rated 3/5 or higher 

Learner Ratings: 

Mean response on all questions 2.5 or higher out of 5 
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Central Cabarrus HS, NC 

Secondary reading, math, language arts 
320 students (credit recovery) 

13 students (test review) 


Study Type: Case study with pre/post gains 


Context Type: Credit Recovery Test 
Review/ Preparation for students who failed 
NCCT. 

Diverse student population, near Charlotte, 
NC 


Program Duration: 2 years 


Key Quantitative Results: 

• Of 320 students who qualified for 
remediation, all have successfully 
recovered credit. 

For 13 students in test review, a significant 
positive relationship between PLATO 
mastery and NCCT test scores: 

• Math: r=.59 (Dec. -May 2000), 
r=.57 (May 2001). 

• Reading: r=.831 (May 2001) 

• Language Arts: r=.833 (May 2001) 


Significance: 

p<.05 on all correlations except Math May 
2001,p=.055 


Effect Size: N/A 


Measure Used: 

NCCT (North Carolina Competency Test 
Pre & post-test 


Instructional Model: 

Supplementary, self-paced, individualized prescription 

Key Qualitative Results: 

In interviews, teachers and administrators were positive about PLATO and believed it 
contributed to student improvement on the NCCT. 

Instructor and Learner Ratings: N/A 
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Career Centers of the Columbus, Ohio Public Schools 

4 centers, including one accredited high school, Ft. Hayes 


Study Type: 
Comparison Study with 
gain comparison 

Context Type: 
Citywide Vocational 
Career Centers 




Program Duration: 2 

Years 


Key Quantitative Results: 


Significance: N/A 


• On the Locating Information test, 1 8% of learners gained one 
level, while 27% of (non-PLATO) learners statewide declined 


Effect Size: N/A 


one Work Keys level 


Measure Used: 


• On the Reading for Information test, 28% of learners 
gained one level, while 4% of (non-PLATO) learners 
statewide declined one Work Keys level. 

• On the Applied Mathematics test, 55% of learners gained 
one level, while statewide only 14% of (non-PLATO) 
learners gained one Work Keys level. 

• At Ft. Hayes, as many as 46% of learners progressed one 
Work Keys level on one of the three tests. 

The pattern of gain was strongest for the Data Processing 

career track, which used only PLATO for core curricula; 

• On Locating Information, 43% (year 1) and 47% (year 2) 
gained one Work Keys level. By contrast, another 
program which did not use PLATO showed a decline of 
5% in year 2. 

• Results for Reading for Information show gains up to 
27% vs. a decline of 16% in a non-PLATO program. 

• Results for Applied Mathematics show gains up to 44% 
vs. a decline of 36% by a non-PLATO program in year 2. 


ACT Work Keys 



Instructional Model: Supplemental & Primary 



Key Qualitative Results: None. Preliminary study. 
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Labette Community College, Parsons, KS 


Study Type: 


AdultAVorkplace Reading and Writing 


Comparison Study 


19 students in Basic Writing I 


Context Type: 


8 students in Basic Writing II 


Rural community college with Title El grant 
to improve academics, especially reading and 
writing, and retention. 




Program Duration: 3 semesters 


Key Quantitative Results: 


Significance: N/A 


• Basic Writing I, with mandatory 
PLATO use, averaged gains of 56 


Effect Size: N/A 


points, with a pass rate of 74%. Basic 
Writing II, with optional PLATO use, 
averaged gains of 1 1 points, greater 
variance, and pass rate of 50%. 

• Basic Reading I, with mandatory 
PLATO use, averaged 46 points gain 
and pass rate gain of 46% (66% total 
pass rate). 


Measure Used: 

Compass reading and writing test pre-/post- 


Instructional Model: 




Primary instmction, self-paced mastery model with active instmctor coaching 
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Forest Grove HS, OR 


Study Type: Comparison study 


High Intensity Learning Lab (HELL) 
208 Secondary Mathematics, 9* Grade 
• 117 PLATO users 


Context Type: 

State test remediation in math 

Students who failed at least 2 parts of OSAT 

in grade 8 

30% on free/reduced lunch, 20% Hispanic 


• 91 Non-PL ATO users 


Program Duration: 1 year 


Key Quantitative Results: 


Significance: /K.OOl 


• HILL students average score increased 
fi’om 228 to 232. These gains were 


Effect Size: .19 


more than two times larger than non- 
PL ATO students. 

• HILL students maintained a statewide 
rank on post-test of 41%ile, vs. a 9- 
point decline fi-om 54%ile to 45%ile for 
non-PLATO/non-HILL students 

• A significant relationship (r=.19, 
p<.05) was identified between PLATO 
module mastery and post-test scores. 


Measure Used: 

Oregon Statewide Assessment Test Math 
Test-retest 



Instructional Model: 

Primary: self-paced individualized mastery model study with teacher in “guide on the side” 
role. 45 minutes in lab during 90 minute block, every other day. 



Key Qualitative Results: 

PLATO users reported feeling more confident about doing well in school. 

Learners found PLATO easy to use, and they tried hard to master the curriculum. 

The FGHS principal and the HELL instmctor believe PLATO contributed to positive results. 

Instructor Ratings: 

All questions rated 4 or higher, except “my students were scheduled to use PLATO for as 
much time as they needed.” = 3. 

Learner Ratings: 

All questions rated 3 or higher, except “the computer makes me nervous” = 2. 
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Fairview Elementary School, Dayton, OH 

Elementary Mathematics 

88 students, 3'^-5* grade (mostly 4* grade). 


Study Type: 

Case study with pre/post measure of gain 


Context Type: 

Title I math class 

55% of students qualify for free/reduced 
lunch. 

84% Black, 15% White, 1% Hispanic & 
Asian students. 

91% average attendance rate 


Program Duration: 3 years 


Key Quantitative Results: 

• On pre-test, 4% of students were rated 
as proficient. End-of-year tests showed 
24% of students proficient. 

• This compares with a school-wide 
average of 12% proficient, and a 
district-wide average of 14% proficient. 

• 81% of students gained. Average pre- 
test was 191 (range 125-227); average 
post-test was 201 (range 148-247). 

• Students at the lower and higher ends 
of math ability gained at about the same 
rate. 


Significance: /?<.001 


Effect Size: 0.5 s.d. 


Measure Used: 

Ohio State Performance test: mathematics 


Instructional Model: 

Pullout program, teacher introduction, students rotate between primary software use 30 
min/period, and 30 min. small group/tutorial work with teacher. 
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Part 



id 



Theory Base of the Reading Curricula 



Each PLATO curriculum receives guidance on content and research on instmction 
from a National Advisory Panel of experts. The panels include nationally 
recognized researchers in curriculum and instmction in the relevant field, senior 
curriculum specialists from school districts and colleges, and PLATO Learning’s 
own curriculum specialists and instmctional design specialists. In addition, 
PLATO Learning draws on its ongoing analysis of curriculum standards in all 
states, Canada and the U.K., as well as the synthesis of standards done by the Mid- 
Central Regional Education Laboratory (McREL). The goal of this planning 
process is to develop a standards-based map of learning outcomes to be taught, 
and to identify research-based “best practices” in teaching and testing each part of 
the curriculum, especially in a computer-based environment. 

Detailed overviews of each curriculum, including a more extensive discussion of 
its underlying research base, its learning outcomes and the features of the products 
are summarized in other PLATO Technical Papers^. For ease of reference, 
however, discussions of the theory base underlying these curricula are excerpted 
here. This Part discusses the reading curricula, and Part 4 discusses the 
mathematics curricula. 



This review of reading research focuses on the instmctional practices that have 
been demonstrated as being effective for beginning readers. These practices are 
the research base for PLATO reading curricula. 

A key finding from current reading research is that there is no “one best way” to 
teach a particular reading skill or capacity. A variety of instmctional methods and 
reading approaches have been shown to be effective, depending upon the 



’ Currently available are; 

Foshay, W. R., E. McEvoy, et al. (2000). Teaching Reading with PLATO: An Overview of the New PLATO 
Reading Solution and How to Use It, rev. 1. Bloomington, MN, PLATO Learning, Inc.: 67. 

, Quinn, B., W. R. Foshay, etal. (2000). Teaching Beginning Reading with PLATO Courseware: An 
Overview of the New PLATO Beginning Reading Solution and How to Use It. Bloomington, MN, PLATO 
Learning, Inc.: 45. 

, Quinn, B., W. R. Foshay, et al. (2000). Teaching Early Mathematics with PLATO Software: An overview 
of the new PLATO elementary mathematics curricula and how to use them. Bloomington, MN, PLATO 
Learning, Inc.: 58. 
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instructional objective and student characteristics. An effective reading program 
will likely involve a mix of instmctional approaches, including direct instmction 
on well-stmctured tasks and problem-solving activities utilizing more open 
assignments and methods. Individual interests and learning needs should be 
recognized in the reading instmction. A broader, more comprehensive review of 
these issues in the reading process can be found in the PLATO Technical Paper, 
Teaching Reading with PLATO. 

While a range of instmctional methods has proven successful in teaching 
begiiming reading, three areas of instmctional focus have proven especially 
effective in helping young, beginning readers learn to read. 

• Alphabetic principles, including phonemic awareness and phonics 
instmction 

• Fluency including reading with accuracy, speed, and expression 

• Comprehension as promoted by vocabulary instmction, text 
comprehension instmction, and teacher preparation and 
comprehension strategies instmction 

The third area, comprehension, is particularly important as readers of any age 
progress past initial decoding; comprehension skills develop throughout education, 
and thus it is as important to develop comprehension strategies at the secondary 
and post-secondary levels as it is at the elementary level. Consequently, it is a 
major objective of the PLATO secondary reading curricula. 

We will discuss some of the instmctional issues regarding each these three areas of 
curriculum focus, and relate them to the PLATO reading curricula. 

Alphabetic Principles 

Beginning readers with little prior print experience need explicit instmction in the 
alphabetic principles of reading: letter identification, phoneme recognition and 
discrimination, phonemic awareness and phonics skills. Two areas of alphabetic 
principles shown to be most important in teaching begiiming readers are phonemic 
awareness and phonics. 

Phonemic Awareness 

Phonemic Awareness (PA) is a vital skill for young children to develop. PA 
means that a child understands that spoken words are made up of a sequence of 
sounds and that these sounds correspond to letters of the alphabet. Understanding 
the alphabetic principle and developing greater phonemic awareness is the first 
step in developing literacy. 
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The importance of PA has been well documented in the educational research (viz., 
Ball & Blachman, 1991; Adams, 1990; & Adams, Forman, Lundberg, & Beeler, 
1998). For instance, PA has been identified as the single best predictor of a child’s 
future reading ability and can account for as much as 50% of the variance in 
reading ability during the first 2 years of instmction (Share, Jorm, Maclean, & 
Matthews, 1984). Blachman (2000), Ehri (1979), Stahl and Murray (1994), and 
Wagner and Torgesen, (1987) also report a strong correlational relationship 
between PA and learning to read. Bradley and Bryant (1983, 1985) have provided 
evidence for a causal relationship between PA and reading ability as well. In a 
recent meta-analysis of PA, the National Reading Panel (2000) determined that the 
effect size of PA on reading instmction (for 52 different published studies that 
included control conditions) was significant (effect size of 0.53). The importance 
of PA is further underscored by research reporting that people who have not been 
taught to read or write have great difficulty performing PA tasks (Morais, 
Bertelson, Cary, & Alegria, 1987), and that poor PA is a strong predictor of 
reading difficulties in the teenage years (Bradley & Bryant, 1983). This has 
important implications for remedial reading at the secondary and adult levels, for 
the small proportion of readers at this level who have poor PA skills. 

Phonemic awareness is taught both explicitly and implicitly in the PLATO 
elementary reading curricula. As suggested by research, PLATO Beginning 
Reading’s explicit phonemic instmction involves the use of phoneme 
manipulation with printed text. The course makes extensive use of a variety of 
strategies to build phonemic awareness: 

• Phoneme isolation, which require recognizing individual sounds in words. 
For example, ‘Mark the words on the screen that start with the /s/ sound.’ 

• Phoneme identity, which requires recognizing the common sound in 
different words. For example, ‘What is the sound that is the same in the 
names Bill, Brett, Becca, and Bob?’ (/b/) 

• Phoneme categorization, which requires recognizing the word with the 
odd sound in a sequence of three or four words, for example ‘Which word 
does not have the /i/ sound of the letter y? fly, yell, try.’ (yell) 

In addition, the program provides extensive experiences with text of all kinds, 
especially with poetry and word games, to help build phonemic awareness. Using 
both direct and indirect methods the courses systematically builds across the grade 
levels a foundation of the most basic phonemic skills to harder and more 
complicated skills in later grades. 

The curriculum makes repeated use of word families and rhyming segments to 
teach reading of new words. This is an example of an analogy phonics 
instmctional strategy. Analogy phonics involves teaching students unfamiliar 
words by analogy to known words (e.g., recognizing that the rhyme segment of an 
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unfamiliar word is identical to that of a familiar word, and then blending the 
known rhyme with the new word onset, such as reading brick by recognizing that - 
ick is contained in the known word kick, or reading stump by analogy to jump). 

Phonics/Phonological Awareness 

Phonological Awareness is a reader’s sensitivity to the patterns of spoken 
language that recur and can be manipulated without respect to the meaning that the 
language patterns ordinarily convey (paraphrase taken from Snow, Bums, & 
Griffin, 1998, p. 111). Phonological awareness is often confused with phonemic 
awareness. Phonemic awareness refers to a child’s understanding of the sounds of 
individual letters and simple blends. Phonological awareness is a more inclusive 
term that refers to a child’s ability to decode representations of more complicated 
combinations of letters and learn spelling. 

Research from several disciplines provides strong evidence for the importance of 
developing phonological skills in learning to read (Adams, 1990; Blachman, 1997; 
Rieben & Perfetti, 1991; Share 1995; Stanovich, 1992). How children perform on 
phonological awareness measures is a powerful predictor of future reading 
achievement (Bryant, Maclean, Bradley, & Crossland, 1990). It has also been 
found that children who lack this phonological insight are likely to be among the 
poorest readers (Blachman, Ball, Black, & Tangel, 1994). Catts (1991, 1993) 
found phonological measures and the naming of objects could predict 83 percent 
of the children’s reading outcomes correctly. Finally, a reciprocal causal 
relationship between early phonological awareness and early literacy acquisition 
has been established in the reading research (Snow, Bums, & Griffin, 1998). 

The National Reading Panel recently conducted a meta-analysis of studies that 
examined phonics instmction. The panel (NRP, 2000) reported that systematic 
phonics instmction, that is, the explicit teaching of a set of specific letter sounds 
and having children read text that provides practice in using these relations to 
decode words, contributed more to children’s growth in reading than unsystematic 
phonics instmction or alternative treatments. There was no single systematic 
phonics program that outperformed the others. 

The PLATO Beginning Reading curriculum is designed to supplement phonics 
instmction. It uses an analogy phonics instmctional strategy, which makes 
repeated use of word families and rhyming segments to teach reading of new 
words. Analogy phonics involves teaching students unfamiliar words by analogy 
to known words (e.g., recognizing that the rhyme segment of an unfamiliar word 
is identical to that of a familiar word, and then blending the known rhyme with the 
new word onset, such as reading brick by recognizing that -ick is contained in the 
known word kick, or reading stump by analogy to jump). 

Since research shows that phonics instmction is both less needed and less effective 
for remediation of adults, it is not a major emphasis of the PLATO secondary 
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reading curriculum. However, the phonics needs of adults are addressed through a 
third-party phonics curriculum available from PLATO Learning. 



Fluency 



Recent education literature suggests that fluency is a critical element in skilled 
reading (NRP, 2000; Snow, Bums, Griffin, 1998). In order for learners to become 
adept readers they must become proficient in the mechanics of reading. 
Specifically, learners need to develop fast and automatic word recognition 
processes, rapidly use punctuation, and group words into meaningfiil units (NRP, 
2000). Developing these skills reduces the cognitive load associated with 
decoding, freeing up resources for understanding the text (comprehension). 

Fluency has been largely ignored in the classroom (Allington, 1983). Ignoring 
fluency has negative consequences. The National Assessment of Educational 
Progress conducted a large study on fluency achievement in U.S. schools (Piimell 
et al., 1995) which concluded that 44% of students sampled were disfluent with 
grade-level materials. The study also reported a relationship between fluency level 
and reading comprehension. 

In its recent meta-analysis examining the effectiveness of methods for teaching 
fluency, the NRP identified an instmctional approach that promotes literacy — 
guided repeated oral reading practice. 

For students to develop fluency with a range of texts, they need adequate practice 
time reading these texts. The amount of practice to develop automaticity in 
children has not been established, but an estimate for developing reading fluency 
in adult learners is about 100 hours of instmction and practice per grade level 
gained (Mikulecky and DAdamo-Weinstein 1991). There is a likelihood that 
young students will require practice time in this same order of magnitude at least. 
Some of the ways students can receive enough practice may be through reading 
aloud to each other in small groups, reading at home to parents or siblings, 
recording their voice as they read, and by the use of interactive computer programs 
for reading practice. 

Fluency building is a major objective of both the elementary and secondary 
PLATO reading curricula. It is supported in the curriculum by previewing a text 
through listening and by having the students read along with a recorded model 
reader. In addition, the interactive design of the courses allows students to reread a 
text several times to become more familiar with it. The amount of reading students 
do in the Beginning Reading and Projects for the New World courses provides 
extensive practice with feedback from the computer. This extensive practice is a 
key element to developing automaticity in reading, a key component in reading 
fluency and comprehension. Similarly, in Vocabulary and Reading 
Comprehension in the secondary curriculum, literally hundreds of hours of 
practice are provided, at graded difficulty levels and with short, medium and long 
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passages of various expository and narrative types, with high interest value to 
secondary and adult readers. 

Comprehension 

Comprehension is a complex and necessary skill that readers of all ages and levels 
need to develop. The ultimate litmus test of a successful reader is whether they 
understand (comprehend) the text they are reading. Consequently, comprehension 
has been described as the “essence of reading” (Durkin, 1993 as cited by NRP 
2000).. The National Reading Panel (NRP, 2000) identified three important areas 
of comprehension: vocabulary instruction, text comprehension instruction, and 
teacher preparation. These areas of comprehension and the panel’s findings for 
each are briefly reviewed below. 



Vocabulaiy 

The NRP provides the following rationale for the importance of vocabulary 
instruction in teaching reading (p. 4-15). 

As a learner begins to read, reading vocabulary encountered in texts is 
mapped onto the oral vocabulary the learner brings to the task. That is, the 
reader is taught to translate the (relatively) unfamiliar words in print into 
speech, with the expectation that the speech forms will be easier to 
comprehend. A benefit in understanding text by applying letter-sound 
correspondences to printed materials only comes about if the resultant oral 
representation is a known word in the learner’s oral vocabulary. If the 
resultant oral vocabulary item is not in the learner’s vocabulary, it will not be 
better understood than it was in print. Thus, vocabulary seems to occupy an 
important middle ground in learning to read. 



A review of the research literature indicates that vocabulary should be taught both 
explicitly and indirectly. Explicit instruction is highly effective for vocabulary 
learning (Tomeson & Aamoutse, 1998; White, Graves, & Slater, 1990; Dole, 
Sloan, & Trathen, 1995; Rinalid, Sells, & McLaughlin, 1997). In addition, the 
more connections that can be made to a specific word, the better it seems to be 
learned. For example, there is empirical evidence indicating that making 
connections with other reading material or oral language in other contexts seems 
to have large effects. 

PLATO curricula combine explicit and indirect vocabulary instruction. For 
example, the secondary Vocabulary and Reading Comprehension curriculum 
includes explicit pre-reading vocabulary instruction using the Vocabulary Builder 
tool, and teachers also can create their own vocabulary lessons with this tool. For 
indirect instruction, many curricula include glossaries with audio for difficult 
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terms, and provide a full, age-appropriate, online dictionary which allows the 
learner to look up the definition of any word on the screen with a few mouse 
clicks. 

Comprehension Instruction 

The second area reviewed by the panel was text comprehension instruction. The 
NRP examined 203 articles that reported on the effectiveness of different 
comprehension instructional practices. The Panel’s analyses identified sixteen 
categories of text comprehension instruction of which seven appear to have a solid 
scientific basis for concluding that these types of instruction improve 
comprehension in non-impaired readers. Some of these types of instruction are 
helpful when used alone, but many are more effective when used as part of a 
multiple-strategy method. The seven effective types of comprehension instruction 
are as follows: 

1. Comprehension monitoring, wherein readers learn how to be aware of 
their understanding of the material; 

2. Cooperative learning, where students learn reading strategies together; 

3. Use of graphic and semantic organizers (including story maps), where 
readers make graphic representations of the material to assist 
comprehension; 

4. Question answering, where readers answer questions posed by the teacher 
and receive immediate feedback; 

5. Question generation, where readers ask themselves questions about 
various aspects of the story; 

6. Story structure, where students are taught to use the structure of the story 
as a means of helping them recall story content in order to answer 
questions about what they have read; and 

7. Summarization, where readers are taught to integrate ideas and generalize 
fi-om the text information. 

The most often used and scientifically based instmctional practices involved 
teaching children how to ask questions when they read, how to monitor their 
comprehension, and how to provide summaries of text. Readers engaged in 
question generation ask themselves who, what, when, where, why, and how 
questions while reading. Readers engaged in comprehension monitoring keep 
track of their comprehension processes and take action when these processes break 
down (Wray, 1994). Readers engaged in summarizing identify the important 
elements of the text and unite those into a coherent whole (NRP, 2000). 
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Reading comprehension is one of the outstanding strengths of the Beginning 
Reading and Projects for the Real World curricula, the PLATO elementary 
reading curricula. Extensive comprehension practice, with exercises using explicit 
and inferential questioning, at multiple levels. The curricula provide extensive 
vocabulary development by direct instruction, hyper-linked definitions of key or 
unusual words and phrases, and extensive experience with words in the context of 
meaningful reading activities. This combination of explicit and indirect instruction 
is an effective mix for building vocabulary and knowledge about the real world 
children live in. 

Comprehension instruction and practice is accomplished through many activities 
and projects that have children apply higher order thinking skills to understanding 
and using the information they read. The courses use the full range of 
comprehension strategies supported by research, including: 

• Comprehension monitoring, where readers learn how to be aware of their 
understanding of the material; 

• Cooperative learning, where students learn reading strategies together; 

• Use of graphic and semantic organizers (including story maps), where readers 
make graphic representations of the material to assist comprehension; 

• Question answering, where readers answer questions posed by the course and 
receive immediate feedback; 

• Question generation, where readers ask themselves questions about various 
aspects of the story; 

• Story structure, where students are taught to use the structure of the story as a 
means of helping them recall story content in order to answer questions about 
what they have read; and 

• Summarization, where readers are taught to integrate ideas and generalize 
fi'om the text information. 

The PLATO secondary reading curriculum includes an even stronger, more 
sophisticated treatment of reading comprehension, in a spiral curriculum structure. 
Initial comprehension strategies are taught in Essential Reading Skills 2, which 
roughly parallels and thus reviews and reinforces the comprehension skills taught 
in the elementary curriculum. Next are the Reading Strategies series of curricula, 
which span grade levels 7-14. These curricula emphasize cognitive strategies for 
reading comprehension across a variety of text types and content areas. A key 
instructional strategy is based on the think-aloud protocol for modeling and 
practicing cognitive strategies, a state-of-the-art approach based directly on the 
research cited above. All of the PLATO reading curricula emphasize both 
referential comprehension and a wide range of inferential skills. The cognitive 
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strategies taught mirror closely the strategies the NRP identified as best supported 
by research. 
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Part 




Theory Base of the Mathematics Curricula 



This review of mathematics research focuses on the instmctional practices that 
have been demonstrated as being effective for learning mathematics, and explains 
how they have been applied to the PLATO mathematics curricula. A key finding 
from this research is that there is no “one best way” to teach a particular math skill 
or capacity. A variety of instmctional methods and teaching approaches have been 
shown to be effective, depending upon the instmctional objective and learner 
characteristics. An effective math program will likely involve a mix of 
instmctional approaches, including direct instmction on well-stmctured tasks and 
problem-solving activities utilizing more open assignments and methods. 
Individual interests and learning needs should be recognized in the math 
instmction. Therefore, the PLATO mathematics curricula use a mix of 
instmctional approaches, and the software is designed to be used in a classroom as 
part of a larger mathematics curriculum, while still being robust and complete 
enough to be used by itself for self-instmctional study. 

The National Council of Teachers of Mathematics (NCTM) standards (1989, 
2000) have been well received by national educational groups, the U.S. 
Department of Education, and the states as they reviewed or formulated new state 
standards, new benchmark tests, and new curriculum materials. The NCTM 
standards have led to less emphasis on skills for their own sake, more on deep 
understanding of important concepts that spiral through curricula and are 
interrelated. Even though a range of methods have proven successful in teaching 
mathematics, across these methods the following areas of instmctional focus have 
proven especially effective in helping young learners to learn mathematics: 

• Skill Modeling and Practice with Feedback 

• Collaborative Learning 

• Computation, Mental Math and Estimation 

• Problem-Solving 

• Active Learning with Real-World Connections 

• Curriculum and Mathematics Connections 

In this part, we will first discuss the influence of NCTM standards on the teaching 
of mathematics, and how PLATO has responded. Then, for each of these 
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instructional foci will be discussed here, with an explanation of how PLATO 
curricula provide this focus. 

Influence of Standards 

Over the last decade, standards developed by NCTM have led to reforms of math 
curricula, textbooks, classroom practices, and state standards. In general, the 
NCTM standards argue for teaching math in a more holistic way. They promote 
teaching concepts, principles and skills in the context of real-world situations and 
teaching the connections among core concepts and principles. 

The NCTM standards have been well received, but they are not without critics. 
On the one hand, the National Science Foundation has supported projects to 
implement the standards and to develop new textbooks and materials. The 
Department of Education has reviewed these and other math reform projects and 
labeled some of them "exemplary," and each state has looked to the NCTM 
standards when formulating their own graduation standards and benchmark tests. 
On the other hand, critics of the NCTM standards and the "exemplary" projects 
include some well-respected (and vocal) educators, mathematicians, and scientists. 
Many of these argue for teaching "math as math" - for retaining the kind of 
abstractions exemplified by a different reform, the "new math" the '60s. Critics 
also include some parents and teachers troubled by the shifts in the newer math 
textbooks away fi’om "basic skills." They argue the reforms go too far, putting too 
much emphasis on the big picture at the expense of algorithmic skills, mental 
math, etc. 

PLATO Learning believes the following trends in math instmction are likely to 
continue and grow in acceptance; 

• In general, less emphasis on skills for their own sake, more on deep 
understanding of important concepts that spiral through curricula and are 
interrelated (fi-action, proportion, ratio, scaling, patterns, fimctions, etc.). 
In other words, skills follow rather than lead. 

• In general, more rich, multi-step problems. For some teachers, instmction 
is problem-driven, which allows the concepts and skills to be taught in 
context. For others, the rich problems come with or after formal 
instmction in concepts, procedures, and skills. 

• In general, more emphasis on how math strands (algebra, geometry, 
measurement, probability and statistics, data collection and analysis, etc) 
are connected - more integration of the strands at each grade level. 

• In algebra, a fimction-based approach rather than an equation-based and 
skills-focused approach. 
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• The addition of topics from these areas: pattern recognition, data collection 
and analysis, probability and statistics, fimctions, and discrete math. 
Topics from these areas are introduced earlier than in the past, so gaps in 
these areas are most apparent for grades 6-8. 

• In the US, a shift in attention toward the bottom 25% of the class. This 
shift is largely driven by individual state standards (based on NCTM 
standards) and the mandated, high stakes tests that determine who passes 
and who graduates. State testing will accelerate changes in math 
instruction for the bottom 25% and add pressure to show good results 
quickly. 

We believe these trends underlie the evolution of most state curriculum standards, 
and we are basing the evolution of our mathematics curricula on these principles. 
The table below summarizes key differences between the approach driven by 
NCTM standards and some traditional math teaching practices. 
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Table 1: Current Approach to Teaching Mathematics 
vs. What Preceded NCTM Reforms 



NCTM Approach 


Traditional Approach 


Learner-based: 

The learner discovers and constructs meaning 
The learner encounters the core concepts and 
principles through investigation. (The teacher 
provides opportunities for investigation and 
facilitates.) 


Teacher-based: 

The teacher/text lay out core concepts and 
principles. The learner practices and applies 
them. 


Integration of math strands: 
algebra, geometry, data analysis, etc., taught 
each year. Connections among math strands 
are explored. 


Separate strands: 

Math strands labeled and taught separately and 
in hierarchy: math fundamentals, algebra, 
geometry, trigonometry, probability, statistics. 


Problem/situation-based approach: 

Students learn concepts and principles as they 
explore a real-world problem or situation. 
Students use a wide range of what they know to 
solve rich problems. 


Modular, skills based: 

Concepts and Principles tend to be taught and 
tested separately with few chances to use them 
to solve a rich problem in context. Problems are 
constrained and tend to have one right answer. 


Function based approach: 

This goes hand-in-hand with the problem-based 
approach. Students observe real world functions 
early in the curricula and know the concept of 
function prior to learning formal notation and 
advanced concepts. 


Equation-based approach: 

Instruction about functions is delayed until 
advanced algebra. It is introduced with numbers 
and variables rather than a real-world situation. 
Real world applications of functions are in a 
separate module. Instruction is heavy with 
notation and terminology. 


Emphasis on Representation - math as 
language: 

Emphasis on many ways to represent problems 
and many ways to solve them using various 
representations (which leads to multiple solution 
paths and sometimes more than one acceptable 
answer). 


Table, graph, equation as ends unto themselves: 
The connections among graphs, tables, and 
equations are taught, but not emphasized and 
not generally connected to real situations. 


Emphasis on higher order thinking/process: 
Learners analyze, interpret, explain their 
reasoning. Learners generate algorithms. 


Emphasis on algorithms, answers: 

More emphasis on algorithms, the right answer, 
the outcome. 


New topics (and at lower levels): 

Data collection and analysis, statistics, 
probability, and discrete math topics are taught 
6-8 and ^12. 


Some coverage of probability and statistics, but 
not for middle school. 


Integration of information about the history of 
mathematics and its contributions. 


Little coverage. 


Technology integrated as tool to allow 
exploration of concepts/principles. 


Some resistance to technology. 



The PLATO elementary curriculum applies these principles in both the tutorial 
curriculum and the interdisciplinary problem solving activities in Projects for the 
Real World. For example, in Projects there is extensive combination of math 
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tasks and topics cross-strand, and problems provide compelling, authentic 
scenarios within which learners can connect what they have learned to what they 
know, as they solve problems. There is a tool-rich work environment, and 
instmctors can use the open-ended environment to adapt the tasks within each 
project to the needs of their learners and their curriculum. 

In the PLATO secondary math curricula, these changes are the principals which 
underlie both new product development and a major upgrade of the math curricula 
which is now under way. Instmctionally, the upgrade stresses: 

• A firm, cross-topic foundation of declarative knowledge 

• Additional emphasis on authentic scenarios and problem solving 

• Learning activities and explanations added and upgraded to enhance 
number sense 

• Greatly expanded treatment of functions and other new topics at lower 
levels of the curriculum 

• Additional emphasis on manipulatives and tools 

• Additional emphasis on high-quality interactions and improved diagnostic 
feedback which directly addresses probable misconceptions 

• Additions of the Investigations learning activities, which include math 
history, open-ended investigation with tools, and opportunities to relate 
math to personal experience. 

In addition, the new architecture features a fresh, new interface and completely 
revised graphics, as well as revised text with controlled readability. 



Now we will turn our attention to a review of research which discusses in more 
detail some of the instmctional issues regarding the areas of instmction identified 
as key by NCTM and supporting research. 

Skill Modeling and Practice with Feedback 

The research basis for PLATO ’s approach to skill modeling with practice and 
feedback is summarized in Part 5’s discussion of Tutorial modules. This mode of 
instmction is commonly used in PLATO curricula to teach declarative knowledge 
(facts, concepts and principles), and to teach well stmctured procedural 
knowledge. Heuristic principles for lesson stmcture are generally consistent with 
principles of direct instmction in the literature (refer to the table in Part 5). 
However, PLATO Learning has adapted and extended the principles according to 
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current cognitive research, for each type of declarative knowledge as well as well- 
structured procedural knowledge. 

Collaborative Learning 

Collaborative learning is an important skill in its own right, and is a useful mode 
for teaching problem-solving (ill-stmctured procedural knowledge) and for 
reinforcing other knowledge types. 

Cook (1993) noted that placing learners in small groups of two to six learners is an 
excellent instmctional strategy for promoting reflective thought and for 
maximizing learner involvement in mathematics interaction. A number of 
researchers in recent years have demonstrated the high degree of learning possible 
when learners can collaborate in learning tasks and when they use their own 
knowledge as a foundation for school learning (Moll, 1989; Moll and Diaz, 1986; 
Palincsar and Brown, 1989; Palinscar, Ramson, and Berber, 1988/89; Brown, 
Palincsar, and Purcell, 1986 ). 

Collaborative classrooms seem to have four general characteristics. 

1 . Shared knowledge among teachers and learners. 

2. Shared authority among teachers and learners. 

3. Teachers as mediators. 

4. Heterogeneous groupings of learners. 

The first two capture changing relationships between teachers and learners. The 
third characterizes teachers' new approaches to instmction. The fourth addresses 
the composition of a collaborative classroom. 

In the PLATO curricula, and especially in mathematics. Problem Solving 
Activities (PSA’s) have been created in the elementary and secondary curricula to 
support collaborative learning. They are open-ended problem-solving activities, 
based on real-world scenarios for the use of mathematics, in which learners work 
together to plan their strategy and execute it-and if necessary, try again, 
depending on the outcome. Thus, the PSA’s are complementary to PLATO 
tutorials. The tutorials themselves, however, also may be used in collaborative 
mode, with pairs of learners working together. 

Computation, Mental Math and Estimation 

Computation, mental math and estimation are closely related topics. Reyes and 
Reyes (1990) provide a clear discussion of their inter-relationship. 
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Do you estimate? Of course you do. Everyone estimates. Research shows that 
estimation is used in real-world problem solving far more than exact 
computation. Furthermore, estimation relates to every important mathematics 
concept and skill developed in elementary school. It is a process that allows 
the user to form an estimate or to judge the reasonableness of a result. The 
NCTM's Curriculum and Evaluation Standards for School Mathematics 
(Standards, 1989) discusses both measurement estimation, for example. 

About how high can you count in one minute? 

About how many beans are in a 1kg bag? 

Is more than 1/2 the area shaded? 

and computational estimation, for example. 

Have you lived 10,000 days? 

I multiplied 48 by 0.27 on my calculator and got 129.6. Can that be right? 

Everything is reduced 35 percent. About how much is saved on the stereo 
in figure 2? 

These questions and the discussion of solutions offer many opportunities for 
developing number sense. 

Estimation includes various interrelated concepts and skills, including mental 
computation, concept development and number sense. In fact, research 
suggests that number sense, mental computation, and estimation are often very 
difficult to separate. Further, the development of any one of these abilities 
often stimulates fiirther growth in the others. 

In the Standards, estimation is highlighted not as an end in itself but as a 
means for helping students "develop insights into concepts and procedures, 
flexibility in working with numbers and measurements, and an awareness of 
reasonable results" (p. 36). The study of estimation should be integrated with 
the study of concepts underlying whole numbers, fi'actions, decimals, and 
rational numbers so that these concepts can be constructed meaningfully by 
the learner. The exploration of a wide range of student-generated estimation 
strategies is recommended. The use of rounding to estimate is singled out for 
less attention in the Standards. Research and common sense clearly document 
that traditional rounding mles (rounding to the nearest ten, hundred, thousand, 
etc.) are often inappropriate and inefficient when estimating. Rather than 
follow rigid mles for estimating, students should be encouraged to use their 
knowledge about number to form estimates that are reasonable in the context 
of the problem. Often this strategy may call for "rounding" to numbers that are 
more compatible with the computation involved. 

The Research Base of PLATO 38 

Copyright ©2002 by PLA TO Learning Inc. 




48 



In grades K-4, the curriculum should include estimation so that students can- 

• explore estimation strategies; 

• recognize when an estimate is appropriate: 

• determine the reasonableness of results; 

• apply estimation in working with quantities, measurement, 
computation, and problem solving.” 



Even though these topics work so well together in a curricular sense, for the 
learner they are not at all the same in the way in which they are processed and 
remembered. Recent brain research has demonstrated that learning math facts is 
very different from applying mathematical reasoning. A recent MIT news release 
(Halbert, 1999), based on work reported in Science by French and MIT 
researchers, reported that: 

learning the multiplication table may be more akin to memorizing a laundry 
list than exercising mathematical skills. Meanwhile, learning to approximate 
how numbers relate to each other seems to be tied to intuition about space. . . 

Through separate studies involving behavioral experiments and brain-imaging 
techniques, the researchers found that a distinctly different part of the brain is 
used to come up with an exact sum, such as 54 plus 78, than to estimate which 
of two numbers is closer to the right answer; exact arithmetic uses a part of the 
brain usually active during verbal memory tasks... This part of the brain, 
while not a primary language area, is activated when subjects have to 
remember verbal material. 

Further, approximating seems to require a more spatial tool, such as a mental 
number line. This spatial tool, which some call number sense, may be the most 
important source of mathematical intuition, although this intuition probably 
also results from interplay between the two brain systems involved. The 
brain-imaging evidence. . . shows that approximate calculations take place in 
the brain's large-scale network involved in visual, spatial and analogical 
mental transformations. . . For years, mathematicians, including Einstein, have 
said that they rely more on mental signs and images than words. 

Halbert wrote that not only were these math activities conducted in different 
locations, but also “the two kinds of math problems were instantaneously assigned 
by the brain to their respective areas, suggesting that the calculation itself, not just 
the decision to perform it, is completed by specific circuits depending on whether 
an exact or approximate result is required.” 
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PLATO mathematics curricula include tutorial modules on mental math and 
estimation, and more are being added. In addition, open-ended investigations 
activities in selected tutorials often place the learner in situations requiring 
estimation and mental math, and the surrounding dialogs reinforce number sense. 
These skills also are an important part of the cognitive strategies needed in 
problem solving activities (PSA’s) in both the elementary and secondary curricula. 
The PSA’s are designed to support multiple solution strategies, and tools and 
occasions for estimation and mental math are built into the learning environment. 

Problem-Solving 

The NCTM standards suggest addressing richer, multi-step mathematics 
problems. One way this can be applied is to have instruction begin with a real 
world example rather than teaching concepts in the abstract. For example, 
graphing an equation is taught to show how real situations can be described by 
graphing data or graphing the equation that describes the data. Standards-based 
approaches to teaching mathematics build in more questions requiring explaining 
the processes and thinking behind the solution, or solutions. Math problem 
solving is designed to provide more modeling, investigating, explaining, and 
showing multiple solutions. 

A recent study shows the benefit of approaching mathematics problem solving 
with a conceptual emphasis. A study of high and low achieving US classes 
(Nowell, Masini, and Quinn) found that teacher instructional practices produced 
measurable effects on learner TIMMS math achievement. In Grade 8 classes, 
teaching practices are related to higher or lower math achievement. Specifically, 
drilling learners on procedures and application of rules is associated with lower- 
achieving classes and focusing on understanding and explaining concepts is 
associated with higher-achieving classes. More teachers in higher-achieving 
classes ask learners to explain the reasoning behind an idea and write equations to 
represent relationships. While these results do not directly test the Standards for 
teaching developed by NCTM, they do show that teaching in a way compatible 
with the Standards is associated with higher math achievement. 

In the PLATO mathematics curricula, problem solving activities (PSAs) using 
advanced architectures are included at both the elementary and secondary levels. 
The Math Problem Solving secondary curriculum, for example, uses real-world 
problem scenarios, open-ended problem solving and interactions in a tool-rich 
environment. Feedback is provided by means of an Intelligent Coach which 
monitors learner performance and provides a dialog on both problem solving 
strategy and tactics. Portfolio assessment is encouraged through a learner log 
which traces the learner’s path through the problem, and allows comparison to an 
expert path. 

In the elementary Projects for the Real World, real-world scenarios provide the 
framework for multidisciplinary problem solving. Learners use the tools of 
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mathematics (as well as language arts, social studies, science, and other 
disciplines) to understand the problem environment and solve the problem. 

Conceptual understanding is built in the tutorial modules, which are designed to 
“wrap around” or support the problem-solving activities. Both tutorials and PSA’s 
can be used in collaborative learning contexts, which help build deep 
understanding. 

Active Learning with Real-World Connections 

Learning does not mean simply receiving and remembering a transmitted 
message; instead, "educational research offers compelling evidence that learners 
learn mathematics well only when they constmct their own mathematical 
understanding" (Mathematical Sciences Education Board, 1990, p. 58). When 
educators begin to see learning as knowledge constmction, they change their 
thinking about curriculum, instmction, and assessment, developing more powerful 
approaches to connecting thinking and mathematics and designing more 
mathematically significant instmctional learning experiences (Cook, 1995). 

Bums (1992) noted that not only is it important to consider the content of the 
mathematics curriculum, it's important to consider how learners leam 
mathematics. Learners need to leam mathematical concepts (declarative 
knowledge) and to see relationships among these concepts (knowledge stmctures). 
Because mathematics concepts are understood only as they relate to the overall 
fi'amework of understanding held by each learner, children must constmct these 
connections through an active process. Such learning experiences are; 

• Hands-on, involving learners in really doing mathematics - experimenting 
first-hand with physical objects in the environment (manipulatives) and 
having concrete experience before learning abstract mathematical concepts 

• Minds-on, focusing on the core concepts and critical thinking processes 
needed for learners to create and re-create mathematical concepts and 
relationships in their own minds 

• Authentic, allowing learners to explore, discover, discuss, and 
meaningfiilly constmct mathematical concepts and relationships in 
contexts that involve real-world problems and projects that are relevant 
and interesting to the learner. 

This philosophy has been applied extensively in the PLATO mathematics 
curricula: 

• Hands-on experience with mathematics is a feature of the new 
mathematics tutorial architecture, and in PLATO PSA’s. Both provide 
manipulatives and powerful tools in scaffolded applications as well as for 
fi'ee-play use. 
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• Minds-on dialog is encouraged throughout in meaningful questioning and 
feedback which addresses misconceptions and stimulates deep 
understanding. The Investigations section of the new tutorial mathematics 
architecture, and the collaborative learning interactions encouraged in 
PSA’s further stimulate reflection. In the secondary PSA’s, dialogs with 
an Intelligent Coach apply intelligent tutoring technology to dialog with 
the learner at the strategy level as well as the tactical level. 

• Authentic scenarios are used extensively to situate explanations, examples, 
practice and assessment. Example scenarios include planning a fishing 
trip, monitoring the progress of a species, managing growth of the stock in 
a greenhouse, and designing a playground. 



Curriculum and Mathematics Integration 

Research has verified the importance of building on learners' prior knowledge 
when helping them leam new concepts. This approach verifies not only the 
importance of articulating learners' math experiences from kindergarten through 
grade 12 but also the importance of aligning learners' math experiences with their 
other experiences both inside and outside school. Educators should keep in mind 
that the development of a child involves multiple settings— the home, the 
neighborhood, the school, and the workplace. People leam and grow in all of these 
settings. Learners of all ages constmct meaning about themselves and their world 
out of personal experiences, including the influences of culture (Caine and Caine, 
1991; Beane, 1995). Learning is enhanced when curriculum and instmction 
integrate learner experiences with the development of meaning. Iran-Nejad, 
McKeachie, and Berliner (1990) state, "The more meaningful, the more deeply or 
elaborately processed, the more situated in context, and the more rooted in 
cultural, background, cognitive, and personal knowledge an event is, the more 
readily it is understood, learned, and remembered" (p. 51 1). 

NCTM gives the following pointers on the need for an articulate, coherent, and 
integrated math curriculum: 

• A well-articulated curriculum challenges learners to leam increasingly 
more sophisticated mathematical ideas as they continue their studies. 

• A mathematics curriculum should be well articulated across the grades. 

• A mathematics curriculum should be coherent. Mathematics comprises 
different topical strands, such as algebra and geometry, but the strands are 
highly interconnected [and] displayed prominently in the curriculum and 
in instmctional materials and lessons... Learners can see how the ideas 
build on, or connect with, other ideas, thus enabUng them to develop new 
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understandings and skills. An effective mathematics curriculum focuses on 
important mathematics — mathematics that will prepare learners for 
continued study and for solving problems in a variety of school, home, and 
work settings. 

Note that both skills and applications such as problem solving are mentioned in 
this list. If learners are to become facile with mathematics, they need automaticity 
with skills and facility with mathematical reasoning. 

The phrase “integration of mathematics instruction” may refer to either of two 
mathematics: (1) mathematics joined with other school subjects, such as math and 
social studies, and (2) different types of mathematics joined with each other, such 
as algebra and geometry. Both of these curricular combinations are legitimate 
ways of intertwining math so that it is better understood and appreciated. 

In the PLATO mathematics curricula, both senses of “integration” have been 
systematically applied. Integration with other areas of study is accomplished 
through interdisciplinaiy problem solving activities (PSAs) in both elementary and 
secondary curricula. In addition, the new secondary math architecture includes 
interdisciplinary topics in the Investigations sections of the tutorials, in the form of 
background information on mathematics and open-ended invitations to explore the 
relationships between math and everyday life. Applied mathematics topics are 
found throughout the curriculum, even including curricula in topics such as 
statistical process control (SPC). 

Integration of the multiple strands of mathematics is accomplished both within the 
PSAs and within the tutorial curricula. The PSAs integrate multiple strands of the 
math curriculum by providing tools and problems which can be solved using a 
combination of strategies (such as equations, graphing and matrices). Tutorial 
curricula are highly modular, and can easily be arranged in a multi-strand, spiral 
curriculum. Furthermore, in the new mathematics architecture, topics such as 
functions and matrix algebra are introduced early in the curriculum, and addressed 
again at higher levels. 

The end result is unusually flexible, interdisciplinary, and comprehensive 
mathematics curricula, comprehensive from elementary number concepts through 
calculus, and rigorously aligned to state and national standards. The curricula 
strike a balance between the important emphases on problem solving and deep 
understanding, while giving adequate attention to computational skills. The 
curricula are designed for use in collaborative learning and conventional 
classrooms, yet are robust enough for self-instructional use in math labs, remedial 
and extended day or home settings. 
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Instructional Design is the field which seeks to apply research on learning, to 
derive prescriptive principles of instruction, and methods for designing learning 
environments which will apply those principles to reliably produce intended 
learning outcomes. PLATO’S instructional design standards are continually 
updated to reflect current research on learning and instruction. In this section, we 
will first summarize the major influences of learning theory on PLATO’S design 
standards. Then we will briefly summarize key standards which apply to each 
type of learning activity in the PLATO system. 



PLATO’S learning theory is based on current cognitive learning research, in 
particular, the ACT’" learning theory proposed by John Anderson (Anderson 
1995). This model distinguishes between declarative and procedural knowledge. 
The following is adapted fi'om (Foshay, Silber et al. 2002) and further explains the 
distinction: 

• Declarative knowledge is knowing that. 

• Procedural knowledge is knowing how. 

These are examples of declarative knowledge: 

• Your phone number. 

• Being able to tell the difference between a table and a tray. 

• Stating that for a car engine to run, it must have air, fuel and 
electrical current for the ignition. 

These are examples of procedural knowledge: 

• Driving to work in the morning by your standard route. 

• Writing a paragraph with correct grammar and spelling. 

• Adding a column of 2-digit numbers. 

• Building a spreadsheet “fi'om scratch” using a software package 
for spreadsheets, to apply a model of environmental carbon 
generation and consumption. 



Basis in Learning Theory 
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• Planning, rearching and writing a persuasive paper on 
environmental carbon policy. 

• Planning a fishing trip’s budget. 

• Designing a copier that can’t jam 

The basic difference between the two types of knowledge is that 
declarative knowledge tells you how the world is, while procedural 
knowledge tells you how to do things in the world. 

Educators who don’t understand this distinction often confuse knowing and doing, 
and make a number of mistakes in designing training: 

• they try to teach (and test) procedural knowledge using strategies which 
are suited for declarative knowledge; 

• they teach declarative knowledge and stop, assuming that the procedural 
knowledge will naturally follow on its own; 

• they try to teach the procedural knowledge without teaching the associated 
declarative knowledge. 

Each of these practices results in ineffective learning and transfer. 

Declarative and procedural knowledge each have a number of types. PLATO 
Learning’s design standards emphasize that it is important to understand the 
different types so designers can recognize them when they plan instmction, and 
use instmctional strategies which are appropriate to each type. 

TVpes of Declarative Knowledge 

Foshay, Silber, et.al. continue: 

There are three types of declarative knowledge: 

1 . Facts, such as names, dates, definitions, formulas, vocabulary, and 
the like. 

2. Concepts, such as groups or categories of things or ideas which go 
by the same name: table, car, love, causation, bigger, weight, mass 
and so on. 

3. Principles, such as “if .then” or causal relationships which explain 
how the world works: gravitational attraction, commutativity, and 
so on. 
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Mental Models 



It’s important to know that the three types of declarative knowledge we’ve 
talked about so far (as well as procedural knowledge) fit together into 
stmctures. These stmctures are really networks of principles (with their 
supporting concepts and facts) and are part of mental models. For 
cognitive psychologists, mental models are the key to learning and using 
knowledge because: 

• they tie together all the declarative knowledge in memory; they are 
the stmctures into which you organize information, put it into 
memory, retrieve it fi'om memory when needed, and learn by 
expanding and restmcturing existing stmctures. 

• they provide the most meaningful application of declarative 
knowledge in isolation. As adults we rarely spout networks of 
facts (unless we are on Jeopardy), or run around finding new 
instances of concepts, but we do fi'equently try to explain how or 
why things happen or work. 

• they form a bridge between declarative knowledge (knowledge 
about) and procedural knowledge (knowing how) - to do 
procedures (other than rote ones), you have to “know how the 
system works” - i.e., have a mental model of the system; 

• the stmcture of the mental model is very different for expert 
performers and for novices. This indicates the importance of 
developing appropriate mental models if the learner is going to 
become a competent performer on the job. 

Therefore, most would argue that for training of adults, the designer must 
not only teach isolated facts, concepts and principles, but must also help 
the learner create the appropriate mental models for optimum stmcturing 
of the information learned for storage, retrieval and application. 

Educators commonly make a number of errors when teaching declarative 
knowledge: 



• They teach all declarative knowledge as if it were fact, by requiring 
learners only to memorize and recall definitions 

• They fail to ask learners to identify or constmct novel examples (for 
concepts), or to predict and explain causal effects (for principles) 

• They teach only the isolated facts, concepts and principles out of 
context, without providing a view of the “big picture” (mental model). 
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and without helping learners to relate what they learn to their own 
mental models of prior knowledge. 

TVpes of Procedural Knowledge 

Foshay, Silber, et.al. continue: 

Procedural knowledge is your ability to string together a series of mental 
and physical actions to achieve a goal. Procedural knowledge is used to 
solve problems and to do things in the world. The type of problem the 
procedural knowledge is used to solve leads to the description of the type 
of procedural knowledge. Problems vary along a continuum based on 
how well they are defined. (Anderson 1995; Jonassen 1997; Jonassen 
2000) 

• At the most precise end are well structured problems. 

• At the far end are ill structured problems 

A term you may sometimes hear for well-defined procedural knowledge is 
rote procedure. You may sometimes hear the term cognitive strategy, and 
metacognitive strategy. They are usually applied to moderately and ill 
structured problem solving. 

Well Stractured Problem-solving 

We consider performing rote procedures to be well-structured problem- 
solving. All elements of the problem situation are known. Well- 
structured problems are usually performed simply by recalling procedures 
and performing them exactly as taught. It’s not even necessary to 
understand why the procedure works. That means that in many situations 
it’s optional to understand underlying principles that explain the why of a 
well-structured procedure. Examples of well-structured problem solving 
include calculating heating and air-conditioning requirements for a 
building, initial decoding in reading, or solving a long division problem. 

Ill Structured Problem-Solving 

In ill structured problems include most of the complex ones our learners 
meet in life. They are typically involved in far transfer, which is the goal 
of education. Examples of ill stmctured problems include almost any kind 
of design activity, be it writing, deriving and proving a formula, 
developing a public policy recommendation, or designing a new building. 
You've probably heard the old saw (a heuristic) that "defining a problem is 
most of solving it." That refers especially to ill stmctured problems. Ill 
stmctured problem solving is the best technical definition of “higher order 
thinking skills.” 
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It should be clear that the continuum of procedural knowledge from well- 
to ill sfructured problems implies a significant range of knowledge and 
skill types. Furthermore, the role played by declarative knowledge, and 
especially by principles, varies considerably. Mastering any kind of 
procedural knowledge involves reorganizing the related declarative 
knowledge for the purpose of problem solving, and building and 
manipulating a mental model of the system being used, repaired, or 
designed. 

Educators often make the error of teaching all problem solving skills as if they 
were well-sfructured. Sometimes, educators get so focused on teaching ill 
sfructured problem solving that they don’t adequately teach the underlying 
declarative knowledge. While some learners can “figure out” the missing 
declarative knowledge, this insfructional error usually leaves most learners with no 
choice but to attempt to learn the problem solving as a rote, well-sfructured 
procedure - much to the frustration of both the learner and the insfructor. This is a 
common weakness of “consfructivisf ’ methodologies. 

Note that this learning theory supercedes Bloom’s Taxonomy of the Cognitive 
Domain (Bloom 1974). While generations of teachers have used Bloom’s 
Taxonomy, more recent theory development, reflected here, provides a more 
theoretically sound, better validated, and more prescriptively usefol framework for 
insfructional design. 



PLATO Learning’s Instructional Design Standards 



PLATO’S insfructional designers are guided by a knowledge base of general 
insfructional design standards. These are research summaries which are 
periodically updated to reflect current “best practice” in cognitive task analysis, 
insfructional strategies, and computer-based insfruction and assessment. The 
guidelines cover all forms of learning activities in the PLATO curricula, including 
tutorials, application/practice, simulation/problem solving, and testing. The 
guidelines describe insfructional design standards for topics such as: 

• Analysis and teaching of declarative knowledge, including facts, concepts 
and principles 

• Analysis and teaching of mental models 

• Analysis and teaching of well-sfructured and ill-sfructured problem 
solving 

• Assessment of all forms of knowledge using online techniques 

• Consfruction of highly interactive computer dialogs 
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• Tailoring instruction for the targeted learners 

• Accessibility standards 

A detailed explanation of these guidelines is beyond the scope of this paper, but 
the following figures summarize a number of the key points. Some design 
standards apply to all types of software, and some apply to specific learning 
activity types: tutorial, application/practice, informational/reference, tool, and 
assessment. Table 2 summarizes key general standards used by PLATO Learning 
(Gagne 1985; Sweller, van Merrienboer et al. 1998; Alessi and Trollip 2001). 



Table 2: PLATO Curricula’s General Design Standards 

• Organization, chunking and pacing shall be clear and understandable to the learners. 

• Internal Consistency of instructional components’ content and knowledge type shall be 
maintained for all declarative and procedural knowledge types. 

• Learner Control type and degree shall be appropriate for the learners and the way they 
will use the learning activity. 

• Flexibility and modular structure shall allow learners and instructors to use the software as 
they want. 

® Interactivity and practice shall be frequent, of the right knowledge type, and have 
feedback on wrong answers which addresses the reason for the error, or explains the 
principles involved. 

• Teacher ’s Role shall be clear, and described in the instructor guide or help system). 

• Learner ’s Role shall be suitable for the instructional model and the classroom. 

In addition, there are design standards which apply to content or information, 
across all learning activities. These are summarized in Table 3.(Jonassen 1996; 
Reigeluth 1999; Reigeluth 1999) 
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Table 3: PLATO’S Standards for Content/Information 



• Content shall be clearly defined. 

• Content shall be complete and accurate for the purpose and the learner. 

• Content shall be aligned to the curriculum standards in both scope and knowledge type. 

• A full range of positive and negative examples and analogies shall be included, which will 
be clear to the learners. 

• Layout and non-text cues shall help learners understand the content’s logical structure and 
direct their attention. 

• Reading level shall be appropriate to the learners. 

• Graphics, visualization and multimedia shall be used in ways which are instructionally 
needed and relevant, and which are appealing to the intended learners 

• Prior knowledge assumptions shall correspond to the intended learners’ 

• Frame of reference, language and examples and imagery shall be appropriate for the 
intended learners. 

• Adequate accessibility shall be assured through interface design and support of assistive 
devices 

• Content shall be free of bias or stereotypes 

Design Standards for Tutorial (Direct) Instruction 

The most ofl;en used measures of learner achievement in the U.S. are scores on 
standardized tests of basic skills. Using this criterion as the desired learner 
outcome, one set of models, labeled direct or explicit instruction (Rosenshine 
1986), has developed overwhelming research support in the past 25 years. Several 
principles of direct instruction, such as more teacher direction and learner-teacher 
interaction, provide the foundation for this approach. These methods of direct 
instruction or focused instruction have been used to teach mathematics and other 
subjects to a wide range of learners regardless of ethnicity, family background, or 
socioeconomic status. For example, both large scale and smaller scale 
experimental research comparing the outcomes of different forms of instruction 
show that; 

1 . Learners who are taught math using direct instruction methods generally 
outperform (both academically and with respect to self-esteem) learners - 
taught with other forms of instruction. 

2. The early gains of children who were taught some subjects with direct 
instruction are sustained in later grades. 
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Caldwell, Huitt, and French (Caldwell, Huitt et al. 1981) provide a direct 
instruction analysis from a transactional perspective. From this viewpoint, both the 
teacher and learner are active participants in the learning process, each with their 
respective responsibilities. At each event of instruction, the transactional 
perspective provides both a recommended teacher activity and a set of alternative 
learner activities. The most important deviation from the other models is that the 
transactional perspective emphasizes teacher/leamer interaction at every event in 
the lesson. It is this principle of frequent, meaningful interaction that is at the heart 
of PLATO’S instruction. 



The following chart (adapted from Slavin 1997) provides a comparison of 
instructional events from several well-known direct instruction models that 
incorporate these principles. For comparison, we have added a column describing 
the PLATO tutorial module. 
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Table 4; Comparison of Direct Instructional Models 



Good & Grouws 
(1979) (Missouri 
Mathematics 
Program) 


Siavin (1994) 


Gagne (1977); 
Gagne & Briggs 
(1979)'* 


Rosenshine 

(1995) 


Hunter (1982) 
(Mastery 
Teaching) 


PLATO 

Tutorials 


1 . Opening 


1 . State learning 
objective and orient 
learners to lesson 


1 . Gain and control 
attention; inform the 
learner of expected 
outcomes 


1 . Provide 
overview 


1 . Objectives; 
provide 

anticipatory set. 


Motivation, 

confidence, 

objective, 

structure of 

content, 

structure of 

presentation 


2. Review 
homework; mental 
computations; 
review 
prerequisites 


2. Review 
prerequisites 


2. Stimulate recall of 
relevant prerequisite 
capabilities 


2. Review, 
checking 
previous day's 
work 


2. Review 


Link (Stimulate 
recall) 


3. Development 


3. Present new 
material 


3. Present the stimuli 
inherent to the 
learning task; offer 
guidance for learning 


3. Present new 
content & skills 


3. Input & 
modeling 


Presentation, 

Examples/ 

modeling. 

Relate (structure 
of content 
summary) 


4. Assess learner 
comprehension 


4. Conduct learning 
probes 


4. Provide feedback 


4. Initial learner 
practice, 
checking for 
understanding, 
feedback & 
correctives 


4. Check 

understanding and 
guided practice 


Practice, 

Feedback, 

Investigations 


5. Seatwork 


5. Provide 

independent practice 




5. Independent 
practice 


5. Independent 
practice 


Application 

lessons 




6. Assess 
performance and 
provide feedback 


5. Appraise 
performance 


6. Frequent 
tests 




Module test, 
prescription to 
review or go on 


6. Homework; 
weekly and 
monthly reviews 


7. Provide distributed 
practice and review 


6. Make provisions for 
transferability; ensure 
retention 


7. Homework; 
weekly and 
monthly reviews 


6. Homework 


Offline practice 
worksheets, 
PLATO Web 
Learning 
Network 



PLATO tutorials form the self-instructional backbone of most major curricula. 
They are used for teaching declarative knowledge, mental models, and well- 
structured procedural knowledge. The tutorial format was the first activity type 



^ As discussed below, the PLATO tutorial strategy is an extension of the Gagne/Briggs model, but with 
independent practice added, and with a number of other enhancements based on current instructional theory. 
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developed on PLATO nearly 40 years ago. It has undergone substantial evolution 
in that time, reflecting advances in theory as well as improved software 
technologies. Table 5 summarizes key tutorial standards (Jonassen 1987; Fleming 
and Levie 1993; Reigeluth 1999; Alessi and Trollip 2001) 

Table 5: PLATO’s Standards for Tutorials 

Tutorials lessons shall: 

• Teach well-defined objectives which accurately describe the content and Taxonomy level 
of what is taught in each lesson. 

• Start each lesson with an orientation/overview which signals structure of content, 
structure of presentation, and establishes motivation and confidence. 

• Present information in small and logically sequenced segments which reflect the 
knowledge structure being taught. 

• Size segments so they contain (for adolescents and adults) up to 5-9 teaching points each, 
in up to 20-30 minutes of study. For difficult content and for elementary-aged children, 3- 
5 teaching points and study times of 10-20 minutes are preferable. 

• Provide guidance throughout the lesson to both the knowledge structure being learned and 
the learning process itself, through suggestions, symbolic cues, and feedback. 

• Include frequent meaningful questions or interactions (not just navigation) for each 
teaching point, keyed to each teaching point and presentation segment, and appropriate to 
the knowledge type(s) being taught. 

• Provide appropriate diagnostic/explanatory feedback on learner responses to the 
questions/interactions, especially for learner errors. 

• Model the right answer if the learner gets “stuck” in an interaction. 

• Jump, based on the learner’s performance and goals, either automatically or by learner 
choice, to additional topics, examples or more practice in varied formats 



Most tutorials are accompanied by additional application practice which reinforces 
the declarative and procedural knowledge taught in the tutorial, and extends the 
complexity, difficulty and contexts beyond those used in the tutorial. Standards 
for practice are summarized in Table 6. 
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Table 6: PLATO’s Standards for Application Practice 



Application Practice Lessons shall: 

• Provide ample opportunities for practicing a particular skill (typically, at least 3 complete 
performances of the skill, across 15 or more questions/interactions) 

• Provide practice in the desired direction of performance (from cue to response) 

• Randomly sequence the elements practiced (except for well-stmctured procedures) 

• Use relevant criteria to judge responses (often correctness and sometimes speed of 
response). 

• Provide immediate and appropriate explanatory feedback based on the criteria for a 
correct answer (“No, that’s not right because. . .”). 

• Provide progressive levels of difficulty, if appropriate to the content and purpose. 
Difficulty is controlled by varying cognitive complexity through factors such as number 
of cues, number of steps, and context. 

• Contain multimedia elements as appropriate to the content. 

• Keep learner interest and motivation in the program in a way which supports the intended 
learning outcome (rather than distracting from it). 

• Provide meaningful interaction between user and the content included in the program 
(not just “click to continue” or interactions relevant only to the game). 

• Use question formats similar to the Module Mastery Tests and state and national 
standards tests 
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Instructional Design Standards for Teaching Problem Solving 

PLATO uses problem solving activities (PSA’s), often called simulations, to teach 
ill-stmctured problem solving (a technical definition of higher-order thinking 
skills). PSA’s use a number of formats, and are sometimes integrated with 
tutorials. They also are developed as stand-alone activities in some curricula. 
PSA’s usually are designed to support either collaborative learning or solo use. 
Table 7 summarizes the key design standards for PSA’s, (van Merrienboer 1997; 
Jonassen 2000) 



Table 7: PLATO’S Standards for Problem Solving Activities (PSA’s) and 

Simulations: 

Simulations and Problem Solving Activities shall: 

• Provide imaginary experience of a real world or fantasy context which reproduces 
those parts of reality needed for transfer to other problems and contexts. 

• Provide a c/earproWem with clear goals. 

• Provide all the necessary rules of the game and simulation-specific information 
needed to complete the task, without having to hunt for them. 

• Include all the key information and action steps used in reality to solve similar 
problems (simplifying as necessary in early problems to provide scaffolding. 

• Make visible phenomena necessary to understanding, even if they are not visible in 
reality. 

• Allow the learner to make the decisions for each key step in solving the problem 
(allowing for scaffolding to skip or simplify steps when appropriate) 

• Provide a plausible range of decision/action options for each key step, without 
unrealistic structure (except as needed to provide scaffolding in early experiences). 

• Provide realistic and plausible consequences for learner responses. 

• Provide necessary tools and information references needed to solve the problem. 

• If teaching of problem solving is a goal, provide coaching and feedback on learners’ 
actions which will stimulate reflection on strategy. 

• Provide variations to allow replay after reflective thought. 

• Enhance the process of transfer of learning by using realistic scenarios, and by 
stimulating reflection on basic principles and strategies 
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PLATO Learning’s designers maintain a clear distinction between software which 
is designed to provide a complete learning environment, when framed and 
supported by an instmctor (the three types discussed so far), and software designed 
to serve as one component of a learning experience created by the instmctor. Two 
software types fall the in the latter category: informational software (which has 
presentations, but typically does not have the highly interactive response and 
feedback typical of tutorials), and tools (which are designed to be used for a 
purpose such as project building, experiments, reports and essays, and the like. In 
PLATO, both types of non-instmctional software exist as stand-alone components, 
and when integrated with a problem solving activity. Standards for informational 
software are in Table 8, and standards for tools are in Table 9. 

Instructional Design Standards for Information and Tools 



Table 8: PLATO’s Standards for Informational Software 

Informational software shall: 

• Include content at the right level of completeness and accuracy for the intended use 
and for the learners, including coimections to primary source material as appropriate. 

• Encourage critical assessment of information sources. 

• Have graphics and multimedia features used to aid interpretation, and convey 
significant information and/or context, if there is a need 

• Provide a search/exploratory environment which provides efficient and effective 
retrieval 

• Be organized using a defined knowledge stmcture which is easily understood by the 
learner 



These standards are based on (Jonassen 1982; Tufte 1998; Tufte 2000; Tufte 

2001 ). 
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Table 9: PLATO’s Standards for Tool Software 

• Support whole, defined tasks as they are defined and described in the curriculum. 

• Make appropriate assumptions about the learner’s goals, skills and prior knowledge. 

• Provide support for instmctionally useful features such as storing work, tracking 
revisions, teacher and peer review/feedback. 

• Scaffold and model tasks as appropriate 

• Link together to share information among tools and among learners as appropriate. 



Instructional Design Standards for Tests 

PLATO Learning’s assessment systems serve a number of purposes: 

• Placement tests provide initial samples of skills taught, in order to make 
inferences about current achievement level and to exempt learners fi'om 
study of skills they already have. They often use a dynamic “tailored 
testing” format, in which “testlets” are administered to learners depending 
on their performance on the test, so learners don’t waste time by being 
tested on knowledge and skills far above or below their level. 

• Progress tests check for mastery of the terminal objective(s) of each 
tutorial module, and are used to regulate progress through the PLATO 
curricula. These are often in the form of short (5-10 item) quizzes which 
accompany each tutorial. 

• Cumulative tests are end-of-course tests which certify attainment of major 
milestones in the curriculum. They can be assembled on a custom basis so 
they test only assigned modules (through the Custom Assessment Test 
utility), or only assigned standards (through the PLATOLink system). 
They can be of any length. 

• Practice tests emulate the form and content of state standards tests, such as 
math and essay writing. They prescribe relevant modules in the PLATO 
curricula. 

Table 10 describes design standards which apply to PLATO tests(Osterlind 1998). 
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Table 10: PLATO’s Standards for Tests 

• All test items are defined by, and referenced to, specified learning objectives and 
curriculum standards. 

• All tests shall be competency-based (rather than norm-referenced). 

• Test items shall correspond in content and knowledge type to their objective. 

• Test items shall be of formats which maximize reliability and validity for the required 
knowledge type being tested. 

• Test items shall be edited using standard stylistic guidelines for each item type. 

• Reading level of test items shall be no greater than for the corresponding courseware, 
and lower if possible. 

• Where random item assignment is used, item pools shall be at least 3 times larger than 
the test length, or (in certain math questions) items will be generated using number 
generators. 

• No judgement on mastery of individual objectives will be based on response to a 
single item. Depending on the test purpose, right answers on 3-8 items per objective 
are required. 

• Domain sampling will be used for cumulative tests only. 

• Progress tests will focus entirely or primarily on the terminal objective(s) for the 
accompanying lessons. 

• Portfolio assessment techniques will be used for assessment of ill-stmctured problem 
solving. 



The research base for these principles is derived fi'om an extensive review of the 
instmctional design literature, and the review is updated continuously. However, 
readers are referred to the following as core texts for instructional design 
principles upon which PLATO Learning’s standards are based. 
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